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Characteristics of California school 
districts in program improvement 



This descriptive analysis provides a 
statistical profile of California's Title I 
school districts in program improve- 
ment. As an independent analysis of 
these districts in the aggregate, it is 
intended to inform the context for 
district improvement as California rolls 
out and refines its district intervention 
strategies. 

Education policymakers and practitioners in 
California, as elsewhere, are actively address- 
ing the program improvement requirements 
of the No Child Left Behind (NCLB) Act of 
2001. They have several years of experience 
with schools in program improvement and 
school improvement approaches. But because 
the district-level element of the education 
accountability system was phased in later, 
less is known about districts that have been 
identified as in need of program improvement, 
about how district performance under state 
accountability rules compares with that of 
their own schools, and about what districts in 
program improvement might have in common 
and how they compare with other districts. 

This report provides a statistical profile of Cal- 
ifornia’s Title I districts in program improve- 
ment. As an independent analysis of these 
districts in the aggregate, it is intended to be 
a useful source of information to California 



decisionmakers about the characteristics of 
districts in program improvement. 

Under the NCLB Act each state must operate 
a two-level education accountability system, 
with one level focused on school performance 
and the other on district performance. Sepa- 
rate accountability calculations are made for 
schools and for districts, which makes it pos- 
sible for individual schools to have a different 
accountability status from their district. To 
make adequate yearly progress, California dis- 
tricts need to meet as many as 46 individual 
requirements. California’s school districts first 
became subject to program improvement in 
2005/06 if they had failed to make adequate 
yearly progress in the two previous years 
in the same content area or accountability 
category. 

A key finding of this study is that the district 
level of California’s accountability system is 
identifying problems that are missed at the 
school level. An examination of how Cali- 
fornia’s 961 Title I districts and their 10,290 
schools did on individual adequate yearly 
progress requirements under the NCLB Act 
reveals that in 2005/06 — the year on which the 
2006/07 program improvement designations 
were based — 207 districts (78 of them in pro- 
gram improvement) failed to make adequate 
yearly progress on at least one requirement, 
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SUMMARY 



even though all their schools did so on the 
same requirement. In addition, 24 districts 
failed to make adequate yearly progress in the 
aggregate, even though not one school in those 
districts failed to do so. Regarding improve- 
ment status, in 2006/07 four districts in pro- 
gram improvement had no schools identified 
for improvement. 

In these cases the districts were held account- 
able for certain student subgroups whose 
performance was not tracked by the school- 
level accountability rules because there were 
too few students in the subgroup at each 
school to meet the minimum subgroup size 
(100 or more) in California for reporting under 
the NCLB Act. This occurred most often for 
the students with disabilities subgroup. While 
such inconsistencies may appear counterintui- 
tive at first, they reflect the effectiveness of 
a two-level accountability system — with the 
district-level system picking up, monitoring, 
and being accountable for students missed by 
the school-level system. 

In 2006/07, 159 (17 percent) of the state’s 961 
Title I school districts were in program im- 
provement. These districts served more than 
2.6 million students, or 42 percent of all public 
school enrollment. Close to half the students in 
districts in program improvement were also en- 
rolled in schools in program improvement (1.2 
million). Compared with the state’s 802 districts 
not identified for program improvement, the 
districts in program improvement were larger 
on average, with more schools and students, 
and were more frequently located in urban 



settings. But although districts in improvement 
had a greater proportion of large districts than 
did districts not identified for improvement, 
some very large districts were not identified for 
improvement. In fact, half of the largest dis- 
tricts in the state (5 of 10) were not identified for 
program improvement in 2006/07. California’s 
districts in program improvement also tended 
to have higher proportions of Hispanic, Black, 
English language learner, and socioeconomi- 
cally disadvantaged students. 

Moving forward, California’s districts in 
program improvement face substantial chal- 
lenges. Only 22 of the 159 districts in program 
improvement made adequate yearly progress 
in 2005/06, and statewide proficiency targets 
were set to increase again in spring 2008. 
Compared with other districts, districts in 
program improvement tend to have more 
English language learner students and stu- 
dents with disabilities and are more likely to 
be held accountable for the test performance of 
these subgroups. Moreover, among all districts 
that are accountable for these two subgroups, 
districts in program improvement have dis- 
proportionately failed to meet the proficiency 
targets for the subgroups. California’s new 
intervention process for districts in program 
improvement, the District Assistance and 
Intervention Team, which will have increased 
prominence in the state’s district improvement 
efforts over the coming years, emphasizes 
improved services to English language learner 
students and students with disabilities. 
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This descriptive 
analysis provides 
a statistical profile 
of California's 
Title I school 
districts in program 
improvement. As 
an independent 
analysis of these 
districts in the 
aggregate, it 
is intended 
to inform the 
context for district 
improvement as 
California rolls 
out and refines its 
district intervention 
strategies. 



WHY THIS STUDY? 

The No Child Left Behind (NCLB) Act of 2001 
calls for a two-part accountability system in each 
state — one part tracking school-level perfor- 
mance and the other district-level performance.' 



One goal of performing separate adequate yearly 
progress calculations at the school and district 
levels is to give educators— from practitioners to 
policymakers — a clear picture of where improve- 
ment efforts are needed so that every student may 
succeed. District-level accountability is the newer 
of the two education accountability elements, 
phased in nationwide beginning with the 2002/03 
school year (see box 1 for a discussion of key con- 
cepts under the NCLB Act). Because Title I school 
accountability policies pre-date the NCLB Act, the 
first California schools were identified for program 
improvement more than a decade ago, in 1996/97, 
but the first California districts were not identified 
until 2004/05. 

Not surprising, therefore, more is known about 
schools in need of improvement, and intervention 
efforts at the school level are largely defined and 
in place. Less understood are districts in program 
improvement. For example, do districts in pro- 
gram improvement — or districts not identified for 
improvement, for that matter — typically have the 
same adequate yearly progress status as the major- 
ity of their schools? If not, why might such incon- 
sistencies occur? Do districts in program improve- 
ment share some characteristics? How do they 
resemble or differ from other districts? In what 
areas do districts most commonly fail to make ad- 
equate yearly progress? Education decisionmakers 
in California and other states in the West Regional 
Educational Laboratory Region have requested 
more information about the characteristics of their 
districts in program improvement. This report 
seeks to provide some of that information. 

California, which operates the largest public 
school system in the country, is piloting the 
District Assistance and Intervention Team (DAIT) 
process, offering guidance, technical assistance, 
monitoring, and support for districts in program 
improvement (California Department of Education 
2007b). The process incorporates a school-level ac- 
ademic survey to measure the presence of program 
components associated with instructional success 
and district-level self-assessments to help districts 
analyze and address the needs of their students, 




2 



CHARACTERISTICS OF CALIFORNIA SCHOOL DISTRICTS IN PROGRAM IMPROVEMENT 



BOX 1 

Accountability under the 
No Child Left Behind Act: 
definitions of key concepts 

The No Child Left Behind (NCLB) 

Act of 2001. The NCLB Act is the 
reauthorization of the Elementary 
and Secondary Education Act, the 
central federal law in K-12 educa- 
tion. At its heart are new provisions 
to drive broad achievement gains, 
eliminate disparities between groups 
of students, and hold states, districts, 
and schools more accountable for 
performance and progress. 

Title I. Title I is the section of the 
NCLB Act governing resources for 
districts and schools serving disad- 
vantaged student populations, includ- 
ing low-performing and high-poverty 
students. It includes accountability 
provisions for the academic perfor- 
mance of all students and subgroups 
of students (ethnic groups, low-income 
students, students with disabilities). 

Adequate yearly progress. States must 
assess students annually in grades 
3-8 and once in high school in 
mathematics and reading/language 
arts tests aligned with state academic 
standards. To make adequate yearly 
progress, schools and districts receiv- 
ing Title I funds must meet participa- 
tion and performance requirements 
on these tests and perform adequately 
on a state-determined “additional 
indicator.” In California this indica- 
tor is the high school graduation rate 
and in K-8, growth in the Academic 
Performance Index, the state’s own 
measure of academic performance. 
Performance requirements rise to 
100 percent proficiency in 2013/14. 
Performance goals are the same for 
all students. 



Annual measurable objectives. Perfor- 
mance requirements in mathematics 
and reading/language arts are ex- 
pressed as the percentage of students 
scoring proficient or above on tests 
aligned with state content standards. 
Though the schedule varies by state, 
annual measurable objectives rise to 
100 percent in 2013/14 in all states. 

Safe harbor provision. The NCLB 
Act’s “safe harbor” provision is an al- 
ternative measure of adequate yearly 
progress. A school or district achieves 
adequate yearly progress by this mea- 
sure if it reduces by at least 10 percent 
over the previous year the percentage 
of students in each subgroup that 
fails to score proficient or above. 

Confidence intervals. Confidence in- 
tervals (calculations of the upper and 
lower limits between which there is 
“confidence” that a school or district’s 
true percentage falls) account for 
statistical uncertainty in the percent- 
age of students scoring proficient. For 
example, if 69 percent of students in a 
school score proficient in mathemat- 
ics, then depending on the number of 
students tested, the lower proficiency 
limit might be 63 percent and the 
upper 75 percent— a result that might 
be high enough to move the school 
above the annual measurable objec- 
tive target in mathematics. Including 
confidence intervals in accountability 
systems is intended to acknowledge 
random measurement errors in 
assessments and hold the entities 
affected harmless. Virtually all states 
use some form of confidence intervals 
(Chudowsky and Chudowsky 2005). 

Another trend is use of confidence 
intervals with safe harbor provisions, 
which relaxes the 10 percent reduc- 
tion rule since confidence intervals 



afford more leniency for smaller sub- 
groups. Eight states (including Cali- 
fornia) added a 75 percent confidence 
interval for safe harbor in 2005; nine 
states did so in 2004 (Chudowsky and 
Chudowsky 2007). 

School and local education agency 
improvement and corrective action. 
Schools and districts that repeat- 
edly fail to make adequate yearly 
progress face increasingly serious 
consequences from the state. In 
general, a school or district not mak- 
ing adequate yearly progress for two 
consecutive years in the same content 
area or accountability category is 
identified as a school in need of 
improvement or a district in pro- 
gram improvement. Though 2004/05 
was the first year that districts were 
subject to improvement nationwide, 
in California implementation was 
delayed a year. A district in program 
improvement that fails to make 
adequate yearly progress for two 
more consecutive years (advancing 
to Year 3 status) becomes subject to 
“corrective action,” which includes 
deferment or reduction of state funds, 
student transfers to a higher per- 
forming school in another district, 
implementation of a new curriculum, 
replacement of district staff, removal 
of schools from the district’s jurisdic- 
tion, appointment of a trustee to run 
the district, and abolition or restruc- 
turing of the district. To exit program 
improvement, a district must meet all 
adequate yearly progress criteria for 
two years in a row. 

To implement these provisions, each 
state had to submit an accountability 
plan for review to the federal govern- 
ment. Appendix C offers an overview 
of school and district accountability 
in California. 
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including English language learner students and 
students with disabilities. All DAIT-guided steps 
are designed to be carried out by a collaborative 
team of DAIT and district officials (California 
Department of Education and California County 
Superintendents Educational Services Association 
2007a, b). 

In considering how best to prepare and deploy 
DAIT teams and other interventions targeted 
to school districts, California’s decisionmakers 
may benefit from a better overall understanding 
of local districts in program improvement. This 
study provides a statistical profile of California’s 
Title I districts in program improvement (see 
box 1). The report addresses two sets of questions. 
One set concerns how the performance of dis- 
tricts in program improvement (as defined under 
California’s district accountability rules) compares 
with the performance of their own schools (as de- 
fined under the state’s school accountability rules). 
The other set of questions concerns common char- 
acteristics of districts in program improvement, 
including how they compare with districts that 



are not identified for improvement. (For details on 
the data sources and methodology used to answer 
these questions, see box 2 and appendix A.) 

California requirements for districts in program 
improvement mirror federal requirements. 
California’s school districts first became subject 
to program improvement in 2005/06 if they had 
failed to make adequate yearly progress during 
2003/04 and 2004/05 in the same content area 
or accountability category. These school districts 
were in a Year 1 status during 2005/06. Districts 
in Year 1 status that did not make adequate yearly 
progress during 2005/06 advanced to Year 2 status 
in 2006/07, while those that did make adequate 
yearly progress remained in Year 1 status. In 
2005/06, 159 (17 percent) of the 961 California 
districts receiving Title I funding had been desig- 
nated for program improvement; 802 (83 percent) 
had not. 1 Another 73 districts did not receive Title 
I funds and were not subject to program improve- 
ment requirements. Districts receiving Title I 
funds account for more than 98 percent of Califor- 
nia’s public school enrollment. (See appendix B for 



BOX 2 

Summary of methods and data 
sources 

To generate a statistical profile of 
California’s districts in program 
improvement, the research team 
acquired online demographic, as- 
sessment, and accountability data 
from the California Department of 
Education. Most data were collected 
in May 2007. 

The research team met with and 
called California Department of 
Education staff to clarify ambiguous 
data elements or findings. Financial 
and staffing data and information 
about the rural-urban characteris- 
tics of districts were downloaded 
from the federal Common Core of 



Data maintained by the U.S. Depart- 
ment of Education, National Center 
for Education Statistics (2007). The 
research team merged datasets to 
link key variables, and district and 
school characteristics were inves- 
tigated using descriptive statistics, 
including frequency distributions, 
cross-tabulations, measures of central 
tendency (generally, the median), and 
measures of variability (for example, 
interquartile range). 

The aim of the study was to docu- 
ment basic descriptive information 
about California districts in pro- 
gram improvement and to compare 
them with districts not identified 
for improvement. The study relied 
on publicly available demographic, 
assessment, accountability, financial, 



and staffing data. Although the re- 
port documents performance differ- 
ences for certain student subgroups 
that are associated with program 
improvement status, the reasons for 
performance differences cannot be 
determined with the data available 
to this study. Students may differ in 
important ways in districts in pro- 
gram improvement and districts not 
identified for improvement, including 
initial achievement before entering 
the district and access to resources 
outside of school that support learn- 
ing and achievement. Thus, the rea- 
sons for the observed differences in 
performance cannot be identified in 
this study. Appendix A offers further 
detail about the research questions, 
data sources, and methodology used 
in this study. 
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CHARACTERISTICS OF CALIFORNIA SCHOOL DISTRICTS IN PROGRAM IMPROVEMENT 



TABLE 1 

Program or improvement status of California's 961 Title I districts, entering 2006/07 school year 



Status 


Number 


Percent of total 


In program improvement 


159 


16.5 


Year 1 status (not making adequate yearly progress for two consecutive years: 
2003/04 and 2004/05 or 2004/05 and 2005/06) 


59 


6.1 


Year 2 status (not making adequate yearly progress for three consecutive years: 
2003/04, 2004/05, and 2005/06) 


100 


10.4 


Not identified for improvement 


802 


83.5 


Made adequate yearly progress in 2005/06 


595 


61.9 


Did not make adequate yearly progress in 2005/06 


207 


21.5 


Total 


961 


100.0 



Source: Authors' analysis based on data from California Department of Education (2007a, g). 



a list of California’s districts in program improve- 
ment in 2006/07, along with student enrollment 
information.) 



(because they have been in program improvement 
for three or more years) in 2006/07, 845 (72 per- 
cent) were in districts in program improvement. 



Of the 159 districts in program improvement, 59 
were in Year 1 of program improvement and 100 
were in Year 2 (table 1). The Year 1 cohort had not 
made adequate yearly progress for two consecutive 
years (in the same content area or accountability 
category), whereas the Year 2 cohort had not made 
adequate yearly progress for three consecutive 
years. An additional 207 districts had not made 
adequate yearly progress during 2005/06 and so 
were at risk of entering program improvement 
in 2007/08 if they did not make adequate yearly 
progress in 2006/07. No California districts were 
subject to “corrective action,” which occurs when 
a district moves into a third year of program 
improvement, because district-level account- 
ability had not yet been in effect for three years in 
California. 2 



In 2006/07, 159 (17 
percent) of California's 
961 Title I school 
districts were in program 
improvement. They 
served more than 
2.6 million students, 
or 42 percent of public 
school enrollment 



In general, California’s districts 
in program improvement oversee 
more schools in improvement than 
do districts not identified for im- 
provement (both per district and 
overall), and their schools tend to 
be facing more severe sanctions 
than schools in improvement in 
other districts. Of the 1,178 Cali- 
fornia schools in corrective action 



FINDINGS OF THE STUDY 

In 2006/07, 159 (17 percent) of California’s 961 Title 
I school districts were in program improvement. 
These districts served more than 2.6 million stu- 
dents, or 42 percent of all public school enrollment. 
Close to half the students in districts in program 
improvement were also enrolled in the 1,439 
schools in program improvement (1.2 million). 
Compared with the state’s 802 districts not identi- 
fied for improvement, the districts in program 
improvement were larger on average, with more 
schools and students, and were more frequently 
located in urban settings. But although districts in 
program improvement had a larger proportion of 
large districts than did districts not identified for 
improvement, some very large districts were not in 
improvement. In fact, half of the largest districts in 
the state (5 of 10) were not identified for program 
improvement in 2006/07. California’s districts in 
program improvement also tended to have more 
student subgroups and have higher proportions 
of Hispanic, Black, English language learner, and 
socioeconomically disadvantaged students. 

A key finding of this study is that the district level 
of California’s accountability system is identifying 
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problems that are missed at the school level. An 
examination of how California’s 961 Title I dis- 
tricts and their schools did on individual adequate 
yearly progress requirements reveals that in 
2005/06 — the year on which the 2006/07 program 
improvement designations were based — 207 
districts (78 of them districts in program improve- 
ment) failed to make adequate yearly progress on 
at least one requirement, even though all their 
schools made adequate yearly progress on the 
same requirement. In addition, 24 districts failed 
to make adequate yearly progress in the aggregate 
even though not one school in those districts failed 
to do so. Regarding improvement status, which is 
based on adequate yearly progress calculations in 
the preceding years, in 2006/07 four districts in 
program improvement had no schools identified 
for improvement. 

In these cases the districts were held accountable 
for certain student subgroups whose performance 
was not tracked by the school-level accountability 
rules because there were not enough students in 
the subgroup to meet the minimum subgroup size 
(100 or more) in California for reporting under 
the NCLB Act. This occurred most often for the 
students with disabilities subgroup. While such 
inconsistencies may appear counterintuitive at 
first, they reflect the effectiveness of a two-level ac- 
countability system — with the district-level system 
picking up, monitoring, and being accountable for 
students missed by the school-level system. 

Moving forward, California’s districts in program 
improvement face substantial challenges. Only 
22 of the 159 districts in program improvement 
made adequate yearly progress in 2005/06, and 
statewide proficiency targets were set to increase 
again in spring 2008. Compared with other dis- 
tricts, districts in program improvement tend to 
have more English language learner students and 
students with disabilities and are more likely to be 
held accountable for the test performance of these 
subgroups. Moreover, among all districts that are 
accountable for these two subgroups, districts in 
program improvement have disproportionately 
failed to meet the proficiency targets for them. 



In 2006/07 four districts 
in program improvement 
had no schools identified 
for improvement 



California’s new interven- 
tion process for districts 
in program improvement, 

DAIT, which will have 
increased prominence 
in the state’s district 

improvement efforts over the coming years, 
emphasizes improved services to English language 
learner students and students with disabilities. 



The district accountability system monitored 
the progress of many students that the school 
accountability system did not have to monitor 

Can a district miss making adequate yearly prog- 
ress even if all its schools make adequate yearly 
progress? Though this may seem counterintuitive, 
separate calculations at the school and district 
levels mean that the answer is yes. The study found 
accountability inconsistencies at multiple levels of 
analysis, from individual adequate yearly progress 
requirements to adequate yearly progress overall 
and district program improvement. To make ad- 
equate yearly progress, California districts need to 
meet as many as 46 individual requirements. 3 In 
2005/06, the year on which the 2006/07 designa- 
tions of districts in program improvement were 
based, 207 (22 percent) of the state’s 961 districts 
failed to make adequate yearly progress on at least 
one adequate yearly progress requirement, even 
though all their schools made adequate yearly 
progress on (or were not accountable for) that 
requirement (table 2). In 24 of these 207 districts 
all the schools met all the requirements or were 
not held accountable for them (not shown in table). 
Collectively, these 207 districts— 78 of them in 
program improvement at the time — enrolled more 
than 1.2 million students (20 percent of the state- 
wide enrollment), and 153 of the districts were in 
city or urban fringe areas. Thus a district may be 
in program improvement even if none or only a 
few of its schools are in program improvement. In 
2006/07 four districts in program improvement 
had no schools in program improvement. 

There are two reasons why districts might not meet 
an adequate yearly progress requirement when all 
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CHARACTERISTICS OF CALIFORNIA SCHOOL DISTRICTS IN PROGRAM IMPROVEMENT 



TABLE 2 

California Title I districts that failed an adequate 
yearly progress requirement but had no schools 
that failed the same requirement, 2005/06 



Category 


Number 
of districts 
or schools 


Percent 
of Title 1 
districts 3 


Districts with district- 


school inconsistencies 


207 


22 


Districts in program 


improvement 


78 


8 


District not identified 
for improvement 


129 


13 


Student enrollment 
in these districts 


1,211,007 


20 



a. N = 961 districts with 6,211,441 students. 

Source: Authors' analysis based on data from California Department of 
Education (2007a). 



their schools do: districts are held accountable for 
students who are not enrolled in a school for a full 
academic year, and individual schools may have 
too few students in a given subgroup to be held 
accountable for its progress. In California schools 
with fewer than 100 students in a subgroup are not 
generally held accountable for the test participation 
or performance of students in that subgroup. But 
when the students in that subgroup at each school 
are aggregated at the district level, the districtwide 
subgroup size may be large enough for an adequate 
yearly progress determination. 



The tendency for 
the district-level 
accountability system 
to identify students not 
meeting a particular 
standard who were not 
identified by the school- 
level accountability 
system was most 
pronounced for students 
with disabilities 



The tendency for the district- level accountability 
system to identify and count students not meeting 
a particular standard who were 
not identified and counted by the 
school-level accountability system 
was most pronounced for stu- 
dents with disabilities (table 3). Of 
California’s 961 districts, 107 (11 
percent) did not meet the English 
language arts annual measurable 
objective proficiency target for 
their students with disabilities 
subgroup in 2005/06, even though 
none of their schools failed to meet 
this target. Such inconsistencies 



resulted almost entirely from the aggregation at a 
district level of subgroups that were too small to be 
counted at individual schools. 

Of the 1,174 schools overseen by these 107 dis- 
tricts, in 2005/06 only 10 had enough students 
with disabilities to be counted for accountability 
purposes, and all 10 met the English language arts 
annual measurable objective proficiency target. 

In other accountability categories for this sub- 
group 46 districts had similar inconsistencies for 
the mathematics proficiency target, 63 districts 
on the English language arts participation rate, 
and 34 districts on the mathematics participation 
rate. Some districts failed more than one require- 
ment. Taken together, 161 California districts (17 
percent) failed to make one of the adequate yearly 
progress requirements for their students with 
disabilities subgroup, even though none of their 
schools failed to make the same adequate yearly 
progress requirement. (The 161 districts represent 
an unduplicated count that cannot be derived 
directly from table 3.) These types of inconsisten- 
cies also were evident, though less common, across 
other adequate yearly progress requirements (see 
table 3). 

Inconsistencies between adequate yearly progress 
outcomes for a district and those for its schools 
are not new. In 2004 the California Department 
of Education sought to address these inconsis- 
tencies, proposing to double to 200 students the 
minimum subgroup size for districts — and to 
raise to 100 students the size for subgroups that 
represented at least 15 percent of the total district 
enrollment (California Department of Education 
2004). However, this proposal was not adopted. 
Thus, California’s adequate yearly progress 
accountability rules at the district and school 
levels continue to yield inconsistent results for 
some districts and their schools. In these cases 
districts are held accountable for students who 
slip through school accountability processes. 
Together, the dual approaches appear compre- 
hensive — counting and being accountable for 
all students. By identifying underperformance 
missed by school accountability, California’s 
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TABLE 3 

Detail for the 207 California Title I districts that failed an adequate yearly progress requirement 
but had no schools that failed the same requirement, 2005/06 (number of districts) 



Adequate yearly progress requirement 


English 
language arts 


Mathematics 


Additional indicator 


Annual measurable objective proficiency target 


Students with disabilities 


107 


46 


na 


English language learner students 


19 


5 


na 


Hispanic students 


9 


5 


na 


Socioeconomically disadvantaged students 


7 


7 


na 


Black students 


5 


6 


na 


School- or districtwide 3 


3 


6 


na 


White students 


2 


4 


na 


Participation rate 


Students with disabilities 


63 


34 


na 


Socioeconomically disadvantaged students 


8 


10 


na 


Black students 


6 


5 


na 


Hispanic students 


3 


5 


na 


White students 


2 


5 


na 


School- or districtwide 3 


1 


2 


na 


English language learner students 


0 


1 


na 


Additional indicator 


Academic Performance Index criteria 


na 


na 


i 


Graduation rate b 


na 


na 


5 



na is not applicable. 

Note: Totals (not shown) exceed 207 because some districts have district-school inconsistencies in multiple categories. 

a. This discrepancy generally reflects students who are not enrolled in a school for a full academic year and so who are not counted at the school level but 
are accounted for at the district level. 

b. This discrepancy reflects high schools too small to be accountable for the graduation rate requirement and students who are not enrolled in a school for a 
full academic year and so who are not counted at the school level but are accounted for at the district level. 

Source: Authors' analysis based on data from California Department of Education (2007a). 



district accountability procedures capture the 
performance and progress of students who might 
otherwise fall through the cracks. 



Districts in program improvement tended to be 
larger and more urban than other districts 

California’s districts in program improvement 
tended to have more schools and higher student 
enrollments than districts not identified for 
improvement. In 2005/06 the median number of 
schools and enrollment in districts in program 
improvement was 12 schools and 7,472 students, 



compared with only 5 schools and 1,213 students 
for other districts. 4 

Collectively, California’s 159 districts in program 
improvement enrolled more than 2.6 million 
students in 2006/07, with just under half of those 
students (1.2 million) also enrolled in schools in 
program improvement (table 4). Another 485,617 
students were in schools in program improvement 
in districts not identified for improvement. 

The state’s districts in program improvement 
tended to be located in more urban settings. In 
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TABLE 4 

Distribution of California students by 2006/07 
school and district program improvement status 





In districts 
in program 
improvement 


In districts 
not 

identified for 
improvement 


Total 


In schools 
in program 
improvement 


1,239,587 


485,617 


1,725,204 


In schools not 
identified for 
improvement 


1,364,819 


3,121,418 


4,486,237 


Total 


2,604,406 


3,607,035 


6,211,441 



Note: The table does not include the 100,995 students enrolled in the 73 
districts that do not receive Title I funds. 

Source: Authors' analysis based on data from California Department 
of Education (2007g) for improvement program status and California 
Department of Education (2007d) for student enrollment. 



2004/05, the most recent year for which data 
were available at the time of writing, 87 percent 
of districts in program improvement were in city 
or urban fringe areas, compared with only 51 
percent of other districts (figure l). 5 However, not 
all large, urban districts were in program im- 
provement. In 2006/07, 5 of the state’s 10 largest 
districts (San Diego Unified, Long Beach Uni- 
fied, Elk Grove Unified, Capistrano Unified, and 
Sacramento City Unified) were not in program 
improvement. 



FIGURE 1 

Distribution of California's Title I districts 
in program improvement and districts not 
identified for improvement by locale, 2004/05 



Districts in 

program improvement 
N= 159 




■ City 
Town 



Districts not 

identifed for improvement 
N = 800 




5% 



■ Urban fringe 

■ Rural 



Note: Because of missing data, districts not identified for program 
improvement total fewer than 802. 

Source: Authors' analysis based on data from U.S. Department of Educa 
tion, National Center for Education Statistics (2007). 



not identified for improvement. For example, in 
2004/05 among districts in program improvement 
the median percentage of Hispanic students (59 
percent) and Black students (3 percent) were both 
about double the medians in districts not identi- 
fied for improvement (24 percent and 2 percent); 
and the median percentage of White students (19 
percent) in districts in program improvement was 
about a third that in other districts (58 percent). 



None of California’s three types of school districts 
(elementary, high school, and unified) was more 
strongly represented among program improve- 
ment districts than any other in 2006/07. The 
distribution of school district types was similar in 
districts in program improvement and in districts 
not identified for improvement: about 54 percent 
were elementary school districts, about 9 percent 
were high school districts, and about 38 percent 
were unified school districts. 



Districts in program improvement had different 
student demographics than did other districts 



The proportions of special populations of students 
tended to differ as well. In 2005/06 the median 
percentages of English language learner students 
and students eligible for free or reduced-price 
lunch were both higher in districts in program 
improvement (table 5). The exception was students 
with disabilities, with median proportions similar 
in both types of districts. 



The share of highly qualified teachers and per student 
staffing and expenditure levels varied little between 
districts in program improvement and other districts 



Districts in program improvement tended to 
have a different ethnic composition from districts 



Across California there was little difference in 
the percentage of core academic subjects 6 taught 
by highly qualified teachers (as defined under 
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TABLE 5 

Student demographics in California's Title I 
districts in program improvement and districts 
not identified for improvement, 2005/06 
(median percentages) 





Districts in 


Districts not 




program 


identified for 


Subgroup 


improvement 


improvement 


Race/ethnicity 


White, not Hispanic 


19.0 


58.2 




(8.1-34.7) 


(33.7-75.8) 


Hispanic 


59.2 


23.6 




(41.9-79.2) 


(9.9-44.8) 


Black, not Hispanic 


3.4 


1.7 




(1.1 -9.9) 


(0.7-4.0) 


Asian 


1.7 


1.6 




(0.8-5. 2) 


(0.6-5. 2) 


American Indian 


0.5 


0.7 




(0. 2-0.9) 


(0. 2-2.0) 


Filipino 


0.7 


0.5 




(0.3-2.0) 


(0-1.4) 


Pacific Islander 


0.3 


0.3 




(0.1 -0.6) 


(0-0.6) 


Special populations 


English language learner 


30.9 


12.1 


students 


(20.3-44.0) 


(4.6-25.2) 


Students eligible for free 


69.7 


44.8 


or reduced-price lunch 


(52.4-81.3) 


(25.5-64.8) 


Students with disabilities 


10.1 


10.1 




(8.5-11.4) 


(7.8-12.2) 



Note: Numbers in parentheses are the spread of values from the 25th 
to the 75th percentiles, which give a sense of how districts vary within 
each group. Because of missing data, districts in program improvement 
sometimes total fewer than 159 and districts not identified for improve- 
ment total fewer than 802: ethnicity: 159 districts in program improve- 
ment, 800 other districts; English language learner students: 159 
districts in program improvement, 721 other districts; students eligible 
for free or reduced-price lunch: 158 districts in program improvement, 
795 other districts; students with disabilities: 159 districts in program 
improvement, 801 other districts. 

Source: Authors' analysis based on California Department of Educa- 
tion (2007c) for race/ethnicity; California Department of Education 
(2007f) for English language learner students; California Department 
of Education (2007e) for students eligible for free or reduced-price 
lunch; and California Department of Education (2007a) for students with 
disabilities. 



the NCLB Act) between districts in program 
improvement and other districts. 7 The median 
per student staffing and per student expenditure 
levels were similar as well. However, in two cases 
the median values for districts not identified for 
improvement fell in the lower or upper quartile 
of the distribution for districts in program 
improvement. California’s districts not identi- 
fied for improvement tended to employ more 
school administrators and fewer instructional 
coordinators and supervisors per 1,000 students 
(table 6). 



Districts in program improvement have 
struggled to meet proficiency targets; other 
districts also face increasing challenges 



Not unexpected, student test performance tended 
to be higher in districts not identified for im- 
provement than in districts in program improve- 
ment, but many of the districts not identified for 
improvement are quite close to being subject to 
program improvement, especially with annual 
measurable objectives set to rise in 2008. The me- 
dian proficiency level (students scoring proficient 
or above) on the spring 2006 English language arts 
test was 33 percent among districts in program 
improvement and 49 
percent among other dis- 
tricts. In mathematics the 
median proficiency level 
was 39 percent among 
districts in program im- 
provement and 51 percent 
among other districts 
(figure 2). 



Student test 
performance tended 
to be higher in districts 
not identified for 
improvement than in 
districts in program 
improvement 



More than 100 districts either did not meet the 
annual measurable objectives or exceeded them 
by no more than 5 percentage points in 2006, a 
slim margin in light of the schedule of annual 
measurable objectives (table Cl in appendix C). 
Specifically, 26 percent of districts in program 
improvement and 10 percent of other districts 
either did not meet the districtwide annual 
measurable objective in at least one subject or 
exceeded it by 5 or fewer percentage points. If 
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TABLE 6 

Median percentages of core academic subjects taught by highly qualified teachers, staffing rates, and 
education expenditures in California's Title I districts in program improvement and districts not identified 
for improvement 





Districts in program 


Districts not identified 


Category 


improvement 


for improvement 


Core academic subjects taught by highly qualified teachers, 2005/06 (percent) 


Highly qualified based on education or training 


60.5 


66.7 




(43.8-86.2) 


(47.7-93.0) 


Highly qualified based on California's High Objective Uniform 


11.5 


17.3 


State Standard of Evaluation (HOUSSE) 


(0.9-35.9) 


(0.0-40.1) 


Number of districts 


159 


788 


Staffing per 1,000 students, 2004/05 


Teachers 


47.5 


49.7 




(45.0-50.0) 


(46.3-55.8) 


School administrators 


2.1 


2.5 




(1 .9-2.5) 


(2.0-3.4) 


District administrators 


0.4 


0.4 




(0.2-0.6) 


(0.0-0.9) 


Instructional coordinators and supervisors 


0.6 


0.3 




(0.4-1 .1) 


(0.0-0. 8) 


Guidance counselors 


0.7 


0.4 




(0.2-1 .3) 


(0.0-1.1) 


Number of districts 


159 


800 


Expenditures per student, 2003/04 (dollars) 


Instructional 3 


4,527 


4,529 




(4,083-4,866) 


(4,122-5,240) 


Noninstructional b 


319 


274 




(252-397) 


(194-393) 


Support services c 


2,273 


2,444 




(2,062-2,602) 


(2,079-3,181) 


Number of districts 


158 


797 



Note: Numbers in parentheses are the spread of values from the 25th to the 75th percentiles, which give a sense of how districts vary within each group. 
Because of missing data, districts identified for program improvement total fewer than 159 and districts not identified for program improvement total fewer 
than 802. 

a. Current expenditures for activities directly associated with the interaction between teachers and students, including teacher salaries and benefits, sup- 
plies (such as textbooks), and purchased instructional services. 

b. Composed predominantly of food services and enterprise operations, such as bookstores and interscholastic athletics. 

c. Incorporates student support services, instructional staff support, general administration support services, school administration support services, opera- 
tions and maintenance, student transportation support services, and other support services. 

Source: Authors' analysis based on data from California Department of Education (2007c), for core subjects; U.S. Department of Education, National Center 
for Education Statistics (2007), for staffing and expenditures per 1,000 students. 
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FIGURE 2 

Median percentages of students scoring proficient 
or above in California's Title I districts in program 
improvement and districts not identified for 
improvement, spring 2006 




English language arts Mathematics 

Note: Because of missing data, districts not identified for improvement 
total fewer than 802. 

Source: Authors' analysis based on data from California Department of 
Education (2007a). 



district-level proficiency rates remain the same, 

85 districts in program improvement (54 percent) 
and 139 other districts (18 percent) will fall short 
of the 2008 annual measurable objective. The 
number of districts that do not make adequate 
yearly progress is likely to grow in 2008/09 and 
beyond, as the annual measurable objective 
targets rise; historical examples of even large 
districtwide gains in student achievement are 
not sufficient to meet the increasingly stringent 
annual measurable objective targets (Linn, Baker, 
and Betebenner 2002). 

These projections of districts not making ad- 
equate yearly progress districtwide are likely 
to underestimate the percentage of districts 
not making adequate yearly progress. Overall 
district-level performance is only one require- 
ment of adequate yearly progress, and it can be 
much higher than the performance of the lowest 
scoring subgroup. Although all the median 
district-level figures displayed in figure 2 were 
above the statewide annual measurable objec- 
tive set for 2004/05, they represented only two 



of the state’s many adequate yearly progress 
criteria, and a district could have failed to make 
adequate yearly progress based on other criteria. 
Furthermore, if these same district-level passing 
rates were achieved in spring 2008, the median 
English language arts proficiency level among 
districts in program improvement (33 percent) 
would fall just below the 2008 statewide annual 
measurable objective (see appendix C). 



While only 22 of 159 (14 percent) of California’s 
districts in program improvement met all their 
adequate yearly progress requirements in 2005/06, 
595 of 802 (75 percent) of other districts did so. 

If these 22 districts in program improvement 
make adequate yearly progress again in 2006/07, 
they will exit program 
improvement. (Adequate 
yearly progress determi- 
nations for 2006/07 were 
released by the California 
Department of Education 
in September 2007.) 



While only 22 of 
159 of California's 



districts in program 
improvement met all 
their adequate yearly 
progress requirements 
in 2005/06, 595 of 802 
of other districts did so 



Most districts that did 
not make adequate yearly 
progress in 2005/06 fell 
short in multiple areas. Of the 207 districts not 
identified for improvement that did not make 
adequate yearly progress in 2005/06, 150 (72 
percent) fell short in more than one area. Of the 
137 districts in program improvement that did 
not make adequate yearly progress in 2005/06, 130 
(95 percent) had multiple problem areas. The most 
challenging requirements by far for those districts 
were the English language arts annual measurable 
objective proficiency targets for English language 
learner students and students with disabilities: 
more than 75 percent of districts in program 
improvement accountable for these subgroups 
failed to meet the requirement for both of these 
subgroups. The next most challenging requirement 
is the mathematics annual measurable objective 
target for the students with disabilities subgroup; 
32 percent of program improvement districts 
accountable for this subgroup failed to meet this 
target (table 7). 
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TABLE 7 

Adequate yearly progress requirements on which California's Title I districts in program improvement fell 
short most frequently in 2005/06 



Adequate yearly progress requirement 


Number of districts 
in program 
improvement held 
accountable for 
requirement 3 


Percentage of 
districts accountable 
that did not meet 
requirement 


English language arts annual measurable objective target for students with 
disabilities subgroup 


132 


78 


English language arts annual measurable objective target for English language 
learner students subgroup 


149 


75 


Mathematics annual measurable objective target for students with disabilities 
subgroup 


133 


32 


Mathematics annual measurable objective target for Black students subgroup 


86 


26 


English language arts test participation for students with disabilities subgroup 


135 


25 


English language arts annual measurable objective target for Hispanic students 
subgroup 


152 


25 


English language arts annual measurable objective target for socioeconomically 
disadvantaged students subgroup 


155 


23 


Districtwide graduation rate 


83 


21 


English language arts annual measurable objective target for Black students 
subgroup 


86 


17 


Mathematics test participation for students with disabilities subgroup 


135 


14 



a. These are the districts in program improvement that had a sufficiently large subgroup size to be held accountable for the specific adequate yearly prog- 
ress requirement in 2005/06. 

Source: Authors' analysis based on data from California Department of Education (2007a). 



Districts in program improvement were accountable 
for more student subgroups than other districts 

Like some schools, some districts did not enroll 
enough students in a particular subgroup to be 
held accountable for the performance of those stu- 
dents, thereby reducing the number of criteria that 
must be met to make adequate yearly progress. 
(Unless the subgroup represents at least 15 percent 
of the overall enrollment, California’s mini- 
mum subgroup size for adequate yearly progress 
purposes is 100 students. Additional details are 
in appendix C.) On average, districts in program 
improvement were accountable for 6.25 subgroups, 
and districts not identified for improvement were 
accountable for 3.94 subgroups; the median num- 
bers of subgroups were 6 and 4 (table 8). 

Among the state’s districts not identified for 
improvement fewer than half (approximately 



TABLE 8 

Number of student subgroups for 
which California Title I school districts 
were held accountable in 2005/06 





Districts in 






program 


Districts not in 


Subgroup value 


improvement 


improvement 


Mean 


6.25 


3.94 


Median 


6.00 


4.00 



Source: Authors' analysis based on data from California Department of 
Education (2007a). 



46 percent) enrolled enough English language 
learner students to be held accountable for their 
performance and test participation in 2005/06, 
and only about 42 percent enrolled enough stu- 
dents with disabilities to be held accountable for 
that subgroup. In contrast, among districts in pro- 
gram improvement almost 94 percent had enough 
English language learner students for which they 
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were held accountable in 2005/06, and almost 83 
percent had an eligible subgroup of students with 
disabilities. 

These discrepancies underscore the difficult 
adequate yearly progress path of the districts in 
program improvement, with their larger stu- 
dent populations and hence greater number of 
subgroups for which to be accountable. Districts 
with more criteria to meet are less likely to 
meet their proficiency targets (Novak and Fuller 
2003). While all districts must meet the needs 
of the diverse students they enroll, districts in 
program improvement, generally, are held ac- 
countable for meeting the needs of more groups 
of students. 

Among districts that are accountable for the per- 
formance of English language learner students and 
students with disabilities, districts not identified 
for improvement have been more successful in 
meeting their annual measurable objective targets 
for those subgroups. For example, of the districts 
held accountable for a students with disabilities 
subgroup in 2005/06, 283 of the 337 (84 percent) 
districts not identified for improvement met the 
English language arts annual measurable objective 



for this subgroup, compared with 29 of 132 (22 
percent) districts in program improvement. 8 A 
similar performance discrepancy was evident for 
districts accountable for the performance of Eng- 
lish language learner students: 283 of the 369 (77 
percent) districts not identified for improvement 
met the English language arts annual measurable 
objective for this subgroup, while only 37 of the 
149 (25 percent) districts in program improvement 
did so (figure 3). 

When the annual measurable objective tar- 
gets last increased, in 2005, more district-level 
subgroups of English language learner students 
fell short of the new proficiency bar. Districts in 
program improvement were particularly affected 
by the change (figure 4, top two panels). The 
English language arts and mathematics trends 
among subgroups of English language learner 
students were typical of the general statewide 
pattern: the annual measurable objective target 
rose, and the percentage of districts that met it 
fell. The students with disabilities subgroup was 
an exception to this trend: both districts in pro- 
gram improvement and districts not identified 
for improvement met the mathematics annual 
measurable objective proficiency target for this 



FIGURE 3 

Percentage of students meeting annual measurable objectives in California's Title I districts accountable 
for subgroups of English language learner students and students with disabilities, spring 2006 




meeting English language arts meeting mathematics meeting English language arts meeting mathematics 

annual measurable objective annual measurable objective annual measurable objective annual measurable objective 



Source: Authors' analysis based on data from California Department of Education (2007a). 
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FIGURE 4 

Annual measurable objective attainment trends among California's Title I districts in program improvement 
and districts not identified for improvement accountable for subgroups of English language learner 
students and students with disabilities, 2004-06 



■ Districts in program improvement 

■ Districts not identified for improvement 



English language learner students 
subgroup trend in English language arts 




100 



75 



50 



English language learner students 
subgroup trend in mathematics 

97.0% 95.4% 




Annual measurable 
objective increase 



25 



2004 



2005 



2006 



Students with disabilities subgroup 
trend in English language arts 



Students with disabilities subgroup 
trend in mathematics 





2004 



2005 



2006 



Source: Authors' analysis based on data from California Department of Education (2007a). 



subgroup at similar or slightly increasing rates 
compared with the previous year in both 2005 
and 2006 (figure 4, bottom right panel). This may 
have been the result of certain adequate yearly 
progress flexibility options granted by the U.S. 
Department of Education, however. 9 And with 
statewide annual measurable objective profi- 
ciency targets that were set to increase again 
in spring 2008, many schools and districts in 
California are likely to have a more difficult time 
making adequate yearly progress. 



IMPLICATIONS FOR RESEARCH AND PRACTICE 

To inform the state’s nascent district-level interven- 
tion efforts, this descriptive study sought to iden- 
tify distinguishing characteristics of California’s 
districts in program improvement, including how 
their accountability-related performance compares 
with that of their own schools. More than a third of 
California’s Title I school districts (344 of 961) did 
not make adequate yearly progress in 2005/06, in- 
cluding all but 22 of the state’s districts in program 
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improvement. Statewide annual measurable objec- 
tive proficiency targets were set to increase again in 
spring 2008, making it more difficult for districts 
to meet adequate yearly progress. Compared with 
other districts across the state, California’s dis- 
tricts in program improvement face particularly 
tough challenges. They are accountable for more 
subgroups (including English language learner 
students and students with disabilities), and they 
tend to have lower rates of proficiency for these 
subgroups. More careful study of the differences 
between student populations in districts in pro- 
gram improvement and other districts, especially 
within the same subgroup designation, could shed 
more light on the differences in academic perfor- 
mance across school districts. 

The use of new primary data (perhaps collected 
through surveys or targeted interviews) could 
offer more nuanced insights on certain key issues, 
including whether and how the education needs 
of students in districts in program improvement 
and other districts differ from each other. A future 
study could also explore differences in support for 
these subgroups, including whether districts not 
identified for improvement are employing useful 



education strategies or offering more specialized 
supports that might be transferable to districts in 
program improvement. Differences in the educa- 
tion needs of students in different types of districts 
could also be studied 
to see whether these 
subgroups face additional 
challenges in districts in 
program improvement. 



Compared with other 
districts across the state 
California's districts in 



program improvement 
are accountable for 



more subgroups, and 
they tend to have lower 
rates of proficiency 
for these subgroups 



Because of the many 
adequate yearly progress 
discrepancies between 
districts and their schools 
related to students with 
disabilities, the state 

might consider focusing its district-level interven- 
tions on coordinating special education services 
across schools. The impact of California’s new 
DAIT process will be of great interest to both 
researchers and state policymakers in the com- 
ing years. Furthermore, as accountability rules 
and intervention approaches will be shaped by 
reauthorization of the NCLB Act, an update of this 
study once these areas are defined in reauthoriza- 
tion would be useful. 
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APPENDIX A 

RESEARCH QUESTIONS, DATA 
SOURCES, AND METHODOLOGY 

The West Regional Educational Laboratory re- 
search team began the study with two guiding sets 
of questions. The first set concerns the character- 
istics of districts in program improvement. Key 
questions included: 

• What is the distribution of districts in 
program improvement by district size and 
density, improvement status, and overall 
achievement levels? 

• What are the demographic features of the 
student populations in districts in program 
improvement? 

• What type and amount of staff support are 
available in districts in program improve- 
ment? How are financial resources spent in 
these districts? 

The second set of questions looked within districts 
and was concerned with the relationship between 
the district and school accountability systems. Key 
questions included: 

• What percentage of students in districts in 
program improvement are enrolled in schools 
in program improvement? What percentage 
of students in districts in program improve- 
ment are enrolled in schools now in corrective 
action? 

• How common is it for a district to be in pro- 
gram improvement when none of its schools 
are in program improvement? How common 
is it for a district to fail to make adequate 
yearly progress when none of its schools fails 
to make adequate yearly progress? 

• What accountability provisions are related 
to inconsistencies between the reasons for 
2005/06 adequate yearly progress classifica- 
tions of districts and schools in those districts? 



To begin to address these questions, the research 
team acquired state data from the comprehensive 
online demographic, assessment, and account- 
ability databases maintained by the California 
Department of Education. The team made a 
handful of phone calls to California Department 
of Education staff to clarify ambiguous data 
elements or findings. In addition, financial and 
staffing data were downloaded from the Common 
Core of Data maintained by the National Center 
for Education Statistics (U.S. Department of Edu- 
cation, National Center for Education Statistics 
2007). 

From the outset it was evident that answering the 
research questions would not require advanced 
statistics. Judicious application of basic statistics, 
including a number of descriptive analyses, would 
be sufficient. The specific statistical approach used 
to address each research question is described 
below. 



Characteristics of identified districts 

For each adequate yearly progress require- 
ment the research team tallied the number of 
districts that met, failed to meet, or were not 
bound by the requirement in 2006/07. Districts 
that did not meet adequate yearly progress for a 
single reason and for multiple reasons were also 
counted. The team summarized the distribution 
of districts in program improvement by status 
and size. 



Characteristics of student populations 

Using simple descriptive statistics, the research 
team examined differences between the distribu- 
tions of districts in program improvement and 
other districts on student demographic character- 
istics, such as 2005/06 ethnic composition and the 
percentages of English language learner students, 
students with disabilities, and socioeconomically 
disadvantaged students. The investigations 
included measures of central tendency (generally 
the median) and measures of variability (such as 
interquartile range). 
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Staff support and expenditures 

Using 2004/05 staffing data from the Common 
Core of Data (U.S. Department of Education, 
National Center for Education Statistics 2007) and 
the California Department of Education (2007a), 
the research team created a merged dataset that 
linked program improvement status to variables 
such as student-teacher ratio, student-guidance 
counselor ratio (for high schools), and teacher 
experience. Additionally, by examining financial 
information from the Common Core of Data, the 
team also explored the percentage of annual bud- 
get spent on supplemental instruction. 



Prevalence of school improvement and corrective action 

By merging state agency lists of schools and dis- 
tricts in program improvement with school-level 
data, the team was able to analyze and describe 
the prevalence of school improvement and school- 
level corrective action within districts in program 
improvement in 2006/07 (California Department 
of Education 2007g). 



Accountability inconsistencies 

The analysis of district-school adequate yearly 
progress inconsistencies involved looking for 
divergent performance at the district and school 
levels in 2005/06 (California Department of Edu- 
cation 2007a). An “inconsistency” was defined as 
the case of a district not making adequate yearly 
progress for any given adequate yearly progress 
requirement but each school within the district 
either meeting the requirement or not being held 
accountable for it. 

First, the research team created a categori- 
cal variable for each adequate yearly progress 
requirement at the district level. District values 
were defined as: 

• None, where at least one school in the dis- 
trict did not meet the requirement, and the 
remaining schools either did not meet the 
requirement or were not accountable for it. 



• Some, where at least one school in the district 
did not meet the requirement, and at least one 
school met the requirement. 

• All, where all schools in the district met the 
requirement, were not accountable for it, or 
both. 

Then, for each adequate yearly progress require- 
ment, these categorical values were cross-tabulated 
with the district’s adequate yearly progress deter- 
mination (Met, Did not meet, or Not applicable) 
and the research team examined situations where 
all schools in the district met the requirement (or 
were not held accountable for it) but the district 
did not meet the requirement, the All-Did Not 
Meet cell of the table. 



Reasons for apparent inconsistencies 

Once the inconsistencies associated with each 
2005/06 adequate yearly progress requirement 
were identified, the research team investigated the 
context associated with the anomalies, explor- 
ing district program improvement status, locale, 
and student enrollment information (California 
Department of Education 2007a, 2007g). 



Additional measures 

The following additional measures were also used. 

• Program improvement status. Lists of districts 
and schools in program improvement were 
obtained from the California Department of 
Education (California Department of Edu- 
cation 2007g). The data represent program 
improvement status entering the 2006/07 
school year. 

• Academic performance. Academic perfor- 
mance for 2003/04, 2004/05, and 2005/06 was 
measured by the statewide assessments used 
in accountability reporting and in the deter- 
minations of adequate yearly progress. Mea- 
sures were accessed through state datasets 
(California Department of Education 2007a). 
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• Budget information. The research team 
obtained district spending on supplemental 
instruction and other major budget catego- 
ries by analyzing budget information in the 
Common Core of Data (U.S. Department of 
Education, National Center for Education 
Statistics 2007). These data reflect spending 
during the 2003/04 school year, one of the re- 
cent years in which the districts in program 
improvement failed to meet adequate yearly 
progress. 



• Sociodemographic variables. The following 
sociodemographic compositional variables 
were examined at the school and district 
level for districts in program improvement: 
enrollment, percentage of students receiving 
free or reduced-price lunch, percentage of 
English language learner students, percent- 
age of students with disabilities, and racial/ 
ethnic composition. These data were obtained 
from state datasets (California Department of 
Education 2007c, 2007d, 2007e, 2007f). 
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APPENDIX B 

THE 159 CALIFORNIA DISTRICTS IN PROGRAM 

IMPROVEMENT IN 2006/07, WITH THEIR 

2005/06 STUDENT ENROLLMENTS 

The 78 districts in bold type missed an adequate 
yearly progress requirement in 2005/06, even 
though none of their schools failed to meet the 
same requirement. The data are from the Califor- 
nia Department of Education (2007g) LEA and 
school program improvement status data files. 

Adelanto Elementary (7,782 students) 

Alisal Union Elementary (7,472 students) 

Alum Rock Union Elementary (13,515 students) 
Alvord Unified (19,869 students) 

Anaheim Elementary (20,690 students) 

Antelope Valley Union High (25,312 students) 
Arvin Union Elementary (3,127 students) 
Atwater Elementary (4,692 students) 

Bakersfield City Elementary (27,890 students) 
Banning Unified (4,906 students) 

Barstow Unified (7,313 students) 

Bellevue Union Elementary (1,738 students) 
Berkeley Unified (9,076 students) 

Bishop Union Elementary (1,326 students) 
Burton Elementary (3,379 students) 

Castaic Union Elementary (3,568 students) 
Centinela Valley Union High (8,000 students) 
Chaffey Joint Union High (24,982 students) 
Chatom Union Elementary (702 students) 

Chico Unified (13,533 students) 

Chowchilla Union High (955 students) 

Coachella Valley Unified (16,418 students) 

Colton Joint Unified (24,715 students) 

Compton Unified (30,233 students) 

Corning Union Elementary (1,976 students) 
Cutler-Orosi Joint Unified (4,058 students) 

Del Norte Co. Office of Education (989 students) 
Del Paso Heights Elementary (1,865 students) 
Delano Union Elementary (7,510 students) 
Desert Sands Unified (27,565 students) 

Dinuba Unified (5,746 students) 

Earlimart Elementary (1,940 students) 

East Side Union High (25,817 students) 

East Whittier City Elementary (8,979 students) 
Eastside Union Elementary (2,914 students) 



Edison Elementary (1,053 students) 

El Centro Elementary (5,857 students) 

El Monte City Elementary (10,881 students) 
Escondido Union Elementary (19,654 students) 
Fairfax Elementary (1,789 students) 

Fallbrook Union High (3,106 students) 

Fort Bragg Unified (1,951 students) 
Franklin-McKinley Elementary (9,659 students) 
Fresno Co. Office of Education (2,052 students) 
Fresno Unified (79,046 students) 

Fullerton Joint Union High (16,299 students) 
Garvey Elementary (6,297 students) 

Greenfield Union Elementary (8,170 students) 
Greenfield Union Elementary (2,449 students) 
Grossmont Union High (24,444 students) 
Hanford Elementary (5,451 students) 

Hayward Unified (22,236 students) 

Healdsburg Unified (2,485 students) 

Hemet Unified (22,368 students) 

Hesperia Unified (20,267 students) 

Hollister School District (6,050 students) 
Hueneme Elementary (8,208 students) 

Jurupa Unified (21,043 students) 

Keppel Union Elementary (3,090 students) 
Kern Union High (35,394 students) 

King City Union Elementary (2,505 students) 
La Habra City Elementary (6,165 students) 
Lagunitas Elementary (307 students) 

Lake Elsinore Unified (20,652 students) 

Lamont Elementary (2,873 students) 

Lancaster Elementary (16,058 students) 
Lawndale Elementary (6,485 students) 

Lemon Grove Elementary (4,147 students) 
Lennox Elementary (7,597 students) 

Little Lake City Elementary (5,022 students) 

Lodi Unified (30,911 students) 

Los Angeles Unified (727,319 students) 

Los Banos Unified (8,768 students) 

Los Nietos Elementary (2,173 students) 

Lost Hills Union Elementary (564 students) 
Lynwood Unified (18,211 students) 

Madera Unified (17,991 students) 

Marysville Joint Unified (9,644 students) 

Mattole Unified (935 students) 

McFarland Unified (3,155 students) 

Merced City Elementary (11,289 students) 
Merced Co. Office of Education (1,747 students) 
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Modesto City Elementary (17,345 students) 
Modesto City High (15,967 students) 

Montebello Unified (35,286 students) 

Monterey Peninsula Unified (11,546 students) 
Moreno Valley Unified (37,019 students) 
Mountain View Elementary (9,741 students) 
Napa Valley Unified (17,408 students) 

Nevada Joint Union High (4,198 students) 
North Sacramento Elementary (4,862 students) 
Oakland Unified (48,135 students) 

Oceanside Unified (21,367 students) 
Ontario-Montclair Elementary (25,376 students) 
Orange Co. Office of Education (8,284 students) 
Oroville City Elementary (3,049 students) 

Oxnard Elementary (16,004 students) 

Pajaro Valley Unified School (19,329 students) 
Palm Springs Unified (23,689 students) 

Palmdale Elementary (22,826 students) 

Parlier Unified (3,867 students) 

Perris Elementary (5,441 students) 

Perris Union High (8,557 students) 

Planada Elementary (788 students) 

Pomona Unified (33,294 students) 

Porterville Unified (13,373 students) 
Ravenswood City Elementary (4,756 students) 
Red Bluff Union Elementary (2,349 students) 
Reef-Sunset Unified (2,584 students) 

Rialto Unified (30,715 students) 

Richland Union Elementary (3,080 students) 

Rio Elementary (4,049 students) 

Romoland Elementary (2,130 students) 
Roseland Elementary (1,718 students) 

Roseville Joint Union High (8,627 students) 
Salinas City Elementary (7,954 students) 
Salinas Union High (13,578 students) 

San Bernardino City Unified (58,661 students) 
San Francisco Co. Off. of Educ. (1,040 students) 
San Francisco Unified (56,236 students) 

San Jacinto Unified (8,322 students) 



San Joaquin Co. Off. of Educ. (2,432 students) 
San Lorenzo Unified (11,613 students) 

San Luis Coastal Unified (7,470 students) 

San Ysidro Elementary (5,087 students) 

Santa Ana Unified (59,310 students) 

Santa Barbara Elementary (5,808 students) 
Santa Maria-Bonita Elementary (12,815) 

Santa Paula Elementary (3,801 students) 

Santa Paula Union High (1,760 students) 

Santa Rita Union Elementary (3,071 students) 
Sonoma Valley Unified (4,914 students) 

South Bay Union Elementary (8,582 students) 
South Whittier Elementary (4,199 students) 
Stockton City Unified (38,936 students) 
Strathmore Union Elementary (752 students) 
Taft City Elementary (2,100 students) 
Tahoe-Truckee Joint Unified (4,304 students) 
Terra Bella Union Elementary (872 students) 
Thermalito Union Elementary (1,455 students) 
Tracy Joint Unified (17,186 students) 

Tulare City Elementary (8,544 students) 

Tulare Joint Union High (4,714 students) 
Vacaville Unified (13,704 students) 

Vallejo City Unified (18,312 students) 

Ventura Unified (17,545 students) 

Victor Valley Union High (12,379 students) 
Vineland Elementary (847 students) 

Visalia Unified (26,105 students) 

Vista Unified (26,207 students) 

Wasco Union Elementary (3,107 students) 
Washington Union High (1,184 students) 
Weaver Union Elementary (2,086 students) 
West Contra Costa Unified (32,184 students) 
West Fresno Elementary (1,504 students) 
Whittier City Elementary (7,029 students) 
Wilsona Elementary (2,017 students) 

Winters Joint Unified (1,940 students) 
Woodlake Union Elementary (1,612 students) 
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APPENDIX C 

SUMMARY OF CALIFORNIA'S DISTRICT 
ACCOUNTABILITY SYSTEM 

This summary describes key components of 
California’s accountability system. It draws on 
the State of California Consolidated State Ap- 
plication Accountability Workbook last amended 
on June 16, 2006 (U.S. Department of Education 
2006). 



The Standardized Testing and Reporting 
program and adequate yearly progress 

California legislated a comprehensive account- 
ability system in 1999, which it later adjusted to 
meet the requirements of the No Child Left Be- 
hind (NCLB) Act. Today, the state’s Standardized 
Testing and Reporting (STAR) program includes 
the assessments used for state accountability as 
well as the elementary and middle school assess- 
ments used to calculate adequate yearly progress 
under the NCLB Act. The California Standards 
Test is used for assessing adequate yearly prog- 
ress in grades 2-8, while the California High 
School Exit Examination (CAHSEE) is used in 
grade 10. 

To make adequate yearly progress in California, 
all districts, schools, and (numerically signifi- 
cant) subgroups must meet or exceed the state’s 
English language arts and mathematics testing 
annual measurable objectives, 10 meet or exceed the 
graduation rate requirement (if a high school or a 
district with at least one high school), demonstrate 
growth on California’s Academic Performance 
Index (API), and demonstrate a participation rate 
of 95 percent or higher on the statewide exams. 
However, a district that fails to meet its annual 
measurable objectives can still make adequate 
yearly progress if it meets all of the following 
conditions: 

• Reduces its percentage of nonproficient 

students by 10 percent over the previous year 
(safe harbor). A confidence interval of 75 per- 
cent is used to make this determination. 



• Attains the annual API status target or dem- 
onstrates growth of at least one API point over 
the previous year. 

• Increases its high school graduation rate (if 
applicable). 

• Has at least a 95 percent participation rate on 
the applicable statewide assessments. 

California’s Title I school districts are identified for 
program improvement when they fail to make ad- 
equate yearly progress for two consecutive years, 
either in the same content area (performance or 
participation rate) or on the same additional indi- 
cator (API or graduation rate), districtwide or for 
the same numerically significant subgroup. 

California applies an additional criterion for districts 
that miss the annual measurable objectives in the 
same content area for two consecutive years. If any 
grade span within the district (elementary, middle, 
or high school) met the grade span annual measur- 
able objective in either of the two years in question, 
the district is not identified for improvement. 

The progression of a district’s program improve- 
ment status can be complicated. To clarify with an 
example, districts entered program improvement 
for the 2006/07 school year based on performance 
and participation data from 2004/05 and 2005/06. 
Districts moved into Year 2 status of program 
improvement in 2006/07 if they were in Year 1 
status in 2005/06 and did not meet adequate yearly 
progress that year. However, a district in program 
improvement Year 1 status in 2005/06 that made 
adequate yearly progress that year remained in 
program improvement Year 1 status (“on hold”) 
in 2006/07. Such a district would exit program 
improvement if it made adequate yearly progress 
again in 2006/07. 



Annual measurable objective proficiency 
baselines and intermediate goals 

California used its 2002 STAR data to establish 
proficiency baselines for elementary and middle 
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schools and used 2001 and 2002 CAHSEE data 
to establish its high school baselines (table Cl). 
According to the state’s accountability workbook, 
“All of these starting points [were] based on the 
percentage of students at ‘proficient’ or above in a 
public school at the twentieth percentile of Cali- 
fornia’s total enrollment among all schools ranked 
by the percentage of students at the proficient or 
advanced levels” (U.S. Department of Education 
2007, p. 27). From these starting points, the state 
set intermediate goals that increased in equal 
increments, reaching 100 percent proficiency by 
spring 2014 (table Cl). 



Adequate yearly progress participation rate 

As in many other states, in California districts 
with enrollments of 50 or more students must 
demonstrate a participation rate of at least 95 
percent on each applicable statewide assessment 
to make adequate yearly progress. Participation 
rates are calculated separately for English lan- 
guage arts and mathematics, and they are deter- 
mined by dividing the number of students tested 
by the number of students enrolled on the first 
day of testing. California considers all students 
who sit for the assessment to be participants, even 



table ci 

California's proficiency baselines and intermediate goals, grades 2-8 and high school (percent) 





Grades 2- 


-8 


High school 


Proficiency baseline and targets 


English 
language arts 


Mathematics 


English 
language arts 


Mathematics 


Spring 2002 baselines/starting points 


Overall 


13.6 


16.0 


11.2 


9.6 


Black, not Hispanic 


19.6 


18.1 


15.4 


10.3 


American Indian 


28.1 


27.8 


25.2 


21.9 


Asian 


51.0 


60.5 


43.4 


52.1 


Filipino 


45.3 


46.6 


37.3 


32.8 


Hispanic 


16.2 


20.2 


12.7 


10.2 


Pacific Islander 


27.6 


29.7 


22.0 


20.2 


White, not Hispanic 


50.7 


48.9 


45.4 


39.4 


Socioeconomically disadvantaged 


16.3 


20.7 


11.3 


10.7 


English language learners 


8.5 


16.5 


9.6 


11.5 


Students with disabilities 


9.7 


12.1 


2.8 


3.5 


Proficiency targets, 2002-14 


2002 Baseline 


13.6 


16.0 


11.2 


9.6 


Spring 2005 


24.4 


26.5 


22.3 


20.9 


Spring 2008 


35.2 


37.0 


33.4 


32.2 


Spring 2009 


46.0 


47.5 


44.5 


43.5 


Spring 2010 


56.8 


58.0 


55.6 


54.8 


Spring 2011 


67.6 


68.5 


66.7 


66.1 


Spring 2012 


78.4 


79.0 


77.8 


77.4 


Spring 2013 


89.2 


89.5 


88.9 


88.7 


Spring 2014 


100.0 


100.0 


100.0 


100.0 



Note: California districts that enroll both primary and secondary school students use an average to determine their intermediate goals. Unified school 
districts and high school districts with grades 7 or 8 have slightly different goals (see California Department of Education 2007i, workbook attachments M 
and N). 

Source: California Department of Education (2007i, workbook attachments G, H, K, L). 
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if they do not respond to enough items to generate 
a result. 



Adequate yearly progress subgroups 

Special education students participate in STAR 
by taking either the grade-level assessment (with 
or without accommodations or modifications) or 
the California Alternate Performance Assessment, 
first administered in spring 2003. Individualized 
Education Plan teams determine how students 
with disabilities will participate in the STAR 
program. California’s English language learner 
students may also take their STAR assessments 
with accommodations. Those who have been in 
the country for one year or less do not have their 
test results counted toward the annual measur- 
able objective calculation, but are counted in the 
participation rate measure. Redesignated fluent 
English proficient students are included in the 
English language learner students subgroup until 
they reach proficiency on the English language 
arts California Standards Test for three years. 

For determining adequate yearly progress, the 
minimum subgroup size in California is either 
100 students with valid test scores or, when the 



subgroup constitutes at least 15 percent of the 
students with valid test scores, 50 students. Test 
scores are considered valid if students have been 
continuously enrolled in the district since the 
preceding October. 



Additional indicator 

California’s additional adequate yearly progress in- 
dicators are API progress for elementary districts 
and the graduation rate for districts that enroll 
high school students. 11 California has created 
growth targets, similar to the annual measurable 
objective schedule, for both additional indica- 
tors. API growth is defined as meeting the annual 
API status target or a demonstrated gain of one 
or more API points. Graduation rate growth is 
defined as improvement of at least 0.1 percent 
over the previous year, improvement of at least 
0.2 percent in the average graduation rate over the 
previous two years, or achievement of an annual 
graduation rate target, which began at 82.8 percent 
and increases at a trajectory similar to the state’s 
annual measurable objectives. A district with high 
school students is identified for program improve- 
ment if it fails to meet one of these graduation rate 
growth criteria for two consecutive years. 
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NOTES 

1 . 

2 . 

3. 



4. 



5. 



Throughout the report, because of missing 
data districts in improvement may sometimes 
total fewer than 159 and districts not identi- 
fied for improvement may sometimes total 
fewer than 802. 

California and the U.S. Department of Educa- 
tion had agreed to a timetable that is one year 
behind that in most other states. 

There are four requirements (English lan- 
guage arts performance and participation, 
mathematics performance and participation) 
that apply to as many as 11 different student 
groups (all students, seven ethnic groups, 
socioeconomically disadvantaged, English 
language learner students, and students with 
disabilities), plus an additional indicator 
(graduation rate for high schools, Academic 
Performance Index for other schools): 4 x 11 
+ 1 + 1 = 46. 

The median values are the 50th percentile for 
the characteristic. Equal numbers of districts 
have higher and lower values. The median is 
used in this report because extreme values — 
such as those associated with Los Angeles 
Unified, by far California’s largest district — 
can skew the average. 

The U.S. Census Bureau uses eight location 
(locale) codes to delineate the urban and 
rural characteristics of school districts. For 
this analysis these codes were merged into 
four, more general density classifications: 
city (“large city” and “mid-size city”), urban 
fringe (“urban fringe of large city” and “urban 
fringe of mid-size city”), town (“large town” 
and “small town”), and rural (“rural, outside 
core-based statistical area” and “rural, inside 
core-based statistical area”). 

The U.S. Code (20 U.S.C. §7801(11)) defines 
“core academic subjects” as English, read- 
ing or language arts, mathematics, science, 



foreign languages, civics and government, 
economics, arts, history, and geography. It 
defines “highly qualified” teachers as hav- 
ing full state certification as a teacher, at 
least a bachelor’s degree, and demonstrated 
subject matter competency in each of the 
academic subjects that they teach. The com- 
plete statutory definition of highly qualified 
also includes components that differentiate 
between new and veteran teachers and among 
elementary, middle, and high school teachers 
(20 U.S.C. §7801(23)). 

7. In accordance with federal guidance Califor- 
nia teachers can obtain highly qualified status 
in several ways. State Superintendent of Public 
Instruction Jack O’Connell has acknowledged 
that the process may conceal important 
differences in teacher qualifications. As he 
explained in August 2007, “While California 
followed federal guidance in the development 
of our Highly Qualified Teacher Requirements 
and California’s High Objective Uniform State 
Standard of Evaluation (HOUSSE) process, we 
know that, unfortunately, in California public 
schools there remains an inequitable distribu- 
tion of highly qualified teachers. Too often, 
schools serving students who are African 
American or Latino have a disproportionate 
number of under-qualified and inexperienced 
teachers and administrators” (California De- 
partment of Education 2007h). The HOUSSE 
process is a federally approved method for 
teachers to demonstrate that they meet highly 
qualified requirements under the NCLB Act. 

8. It should be noted that some of the districts in 
improvement had that status because of the 
large numbers of students with disabilities 
that had scored below proficient. 

9. In May 2005 the U.S. Department of Educa- 
tion began granting states some flexibility 
for the students with disabilities subgroup. 
Schools or districts that failed to make 
adequate yearly progress solely because this 
subgroup did not reach its annual measurable 
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objective target could add 20 percent- 
age points to the proficiency score for this 
subgroup. Furthermore, in previous years 
the results of students tested with modifica- 
tions were counted in the adequate yearly 
progress participation rate as “tested” and in 
the percentages proficient as “not proficient.” 
Beginning in spring 2006, however, these 
student records were counted in California’s 
adequate yearly progress participation rate 
as “not tested” and were not counted in the 
percentage proficient calculation (California 
Department of Education 2006a). 



10. Small schools and districts with fewer than 
100 valid scores have their annual measur- 
able objectives adjusted through a 99-percent 
confidence interval. 

11. California uses a graduation rate calculation 
that corresponds to the National Center for 
Education Statistics’ four-year completion 
rate: the number of four-year high school 
completers divided by the sum of the four- 
year completers plus the total number of 
dropouts, aggregated over the four-year 
period. 
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